The Revival of Essay-Type Questions in Medical Education: Harnessing Artificial Intelligence and Machine Learning.

OBJECTIVE: To analyse and compare the assessment and grading of human-written and machine-written formative essays. STUDY DESIGN: Quasi-experimental, qualitative cross-sectional study. Place and Duration of the Study: Department of Science of Dental Materials, Hamdard College of Medicine & Dentistry, Hamdard University, Karachi, from February to April 2023. METHODOLOGY: Ten short formative essays of final-year dental students were manually assessed and graded. These essays were then graded using ChatGPT version 3.5. The chatbot responses and prompts were recorded and matched with manually graded essays. Qualitative analysis of the chatbot responses was then performed. RESULTS: Four different prompts were given to the artificial intelligence (AI) driven platform of ChatGPT to grade the summative essays. These were the chatbot's initial responses without grading, the chatbot's response to grading against criteria, the chatbot's response to criteria-wise grading, and the chatbot's response to questions for the difference in grading. Based on the results, four innovative ways of using AI and machine learning (ML) have been proposed for medical educators: Automated grading, content analysis, plagiarism detection, and formative assessment. ChatGPT provided a comprehensive report with feedback on writing skills, as opposed to manual grading of essays. CONCLUSION: The chatbot's responses were fascinating and thought-provoking. AI and ML technologies can potentially supplement human grading in the assessment of essays. Medical educators need to embrace AI and ML technology to enhance the standards and quality of medical education, particularly when assessing long and short essay-type questions. Further empirical research and evaluation are needed to confirm their effectiveness. KEY WORDS: Machine learning, Artificial intelligence, Essays, ChatGPT, Formative assessment.

Exploring ChatGPT's abilities in medical article writing and peer review.

AIM: To evaluate the quality of ChatGPT-generated case reports and assess the ability of ChatGPT to peer review medical articles. METHODS: This study was conducted from February to April 2023. First, ChatGPT 3.0 was used to generate 15 case reports, which were then peer-reviewed by expert human reviewers. Second, ChatGPT 4.0 was employed to peer review 15 published short articles. RESULTS: ChatGPT was capable of generating case reports, but these reports exhibited inaccuracies, particularly when it came to referencing. The case reports received mixed ratings from peer reviewers, with 33.3% of professionals recommending rejection. The reports' overall merit score was 4.9±1.8 out of 10. The review capabilities of ChatGPT were weaker than its text generation abilities. The AI as a peer reviewer did not recognize major inconsistencies in articles that had undergone significant content changes. CONCLUSION: While ChatGPT demonstrated proficiency in generating case reports, there were limitations in terms of consistency and accuracy, especially in referencing.

Heart Rate Measurement Using the Built-In Triaxial Accelerometer from a Commercial Digital Writing Device.

Currently, wearable technology is an emerging trend that offers remarkable access to our data through smart devices like smartphones, watches, fitness trackers and textiles. As such, wearable devices can enable health monitoring without disrupting our daily routines. In clinical settings, electrocardiograms (ECGs) and photoplethysmographies (PPGs) are used to monitor heart and respiratory behaviors. In more practical settings, accelerometers can be used to estimate the heart rate when they are attached to the chest. They can also help filter out some noise in ECG signals from movement. In this work, we compare the heart rate data extracted from the built-in accelerometer of a commercial smart pen equipped with sensors (STABILO's DigiPen) to standard ECG monitor readouts. We demonstrate that it is possible to accurately predict the heart rate from the smart pencil. The data collection is carried out with eight volunteers writing the alphabet continuously for five minutes. The signal is processed with a Butterworth filter to cut off noise. We achieve a mean-squared error (MSE) better than 6.685 × 10-3 comparing the DigiPen's computed Δt (time between pulses) with the reference ECG data. The peaks' timestamps for both signals all maintain a correlation higher than 0.99. All computed heart rates (HR =60Δt) from the pen accurately correlate with the reference ECG signals.

Evaluation of Large Language Model Performance and Reliability for Citations and References in Scholarly Writing: Cross-Disciplinary Study.

BACKGROUND: Large language models (LLMs) have gained prominence since the release of ChatGPT in late 2022. OBJECTIVE: The aim of this study was to assess the accuracy of citations and references generated by ChatGPT (GPT-3.5) in two distinct academic domains: the natural sciences and humanities. METHODS: Two researchers independently prompted ChatGPT to write an introduction section for a manuscript and include citations; they then evaluated the accuracy of the citations and Digital Object Identifiers (DOIs). Results were compared between the two disciplines. RESULTS: Ten topics were included, including 5 in the natural sciences and 5 in the humanities. A total of 102 citations were generated, with 55 in the natural sciences and 47 in the humanities. Among these, 40 citations (72.7%) in the natural sciences and 36 citations (76.6%) in the humanities were confirmed to exist (P=.42). There were significant disparities found in DOI presence in the natural sciences (39/55, 70.9%) and the humanities (18/47, 38.3%), along with significant differences in accuracy between the two disciplines (18/55, 32.7% vs 4/47, 8.5%). DOI hallucination was more prevalent in the humanities (42/55, 89.4%). The Levenshtein distance was significantly higher in the humanities than in the natural sciences, reflecting the lower DOI accuracy. CONCLUSIONS: ChatGPT's performance in generating citations and references varies across disciplines. Differences in DOI standards and disciplinary nuances contribute to performance variations. Researchers should consider the strengths and limitations of artificial intelligence writing tools with respect to citation accuracy. The use of domain-specific models may enhance accuracy.

Expanding capacity for publication in nursing: Experiences from a writing for publication group.

Nurses have valuable knowledge and expertise to share. Yet, for a variety of reasons, many nurses do not write for publication. Members in one Sigma Theta Tau International chapter requested information about publishing so a writing for publication program (WPP) was convened. Ten nurses from diverse clinical and academic backgrounds participated. The goal of the WPP was to support a small group of nurses to advance knowledge and develop practical skills through the development of a manuscript with mentorship from doctorally-prepared nurses with publishing experience. The anticipated effect was that participants would share what they learned with colleagues or mentor others to publish in the future. Beginning with informational sessions to lay the foundation for writing and publishing, the WPP included biweekly, two-hour online sessions over a seven-month period whereby individual and group writing with embedded peer and WPP leader feedback occurred. WPP participants gained proficiency in searching online databases, synthesizing published literature, and working as a member of a writing team. The group successfully published a manuscript based on a topic of interest. This current article describes the structured support and mentorship provided during the WPP with recommendations for overcoming publication barriers commonly described in the literature.

Historia de la escritura como herramienta para la práctica de la psiquiatría y la salud mental.

La escritura como la conocemos hoy en día es un desarrollo relativamente nuevo en la historia cultural de la humanidad. Más actual aún es la masividad de la alfabetización de la población. Desde los orígenes de la psiquiatría, la escritura forma parte inherente de ésta en la bibliografía médica pero también como una herramienta aplicada en la práctica en diversos formatos, tanto con fines diagnósticos como terapéuticos. No obstante, frente a las herramientas clásicas de comunicación, como el habla, y en la psiquiatría infantil, también el juego y el dibujo, la escritura parece haber sido relegada. En esta investigación bibliográfica, que no tendría antecedentes en su tipo, se realizó un recorrido histórico de la temática desde el surgimiento de la psiquiatría como especialidad hasta nuestros días; donde hechos como las Guerras Mundiales, la evolución del concepto de discapacidad o la pandemia por COVID-19 parecen haber jugado un papel clave para delimitar variables que caracterizan, al menos, tres períodos históricos y presumir un cuarto en plena gestación. El soporte tecnológico con el que se escribe y el uso y formato dado a la escritura resultaron las principales variables de los distintos períodos. Siendo ésta una época de muy alta demanda en Salud Mental, desafiante en su accesibilidad y la masividad en el uso de la escritura a partir de tecnologías de la información, esta investigación representaría un aporte significativo al desarrollo de la escritura como herramienta para la práctica de la psiquiatría y la salud mental.

Does it Appear to 'Resemble' Reality? on the Ethics of Psychoanalytic Writing.

This paper explores the intricate nexus of writing and psychoanalysis by addressing a key question: In what and how many directions should analytic writing be ethical? The author structures the argument across three axes. First, in an introduction, writing's role as a psychoanalytic invariant is emphasized. Then, an exploration ensues, delving into writing as praxis, navigating complex technical choices, from micro- to macro-perspectives in clinical vignettes, their autobiographical essence, their relevance as models for theory, self-revelation, etc. Lastly, a succinct epilogue considers the relationship between aesthetics and ethics in psychoanalytic writing.

Implementation and evaluation of an interprofessional prescription writing workshop with a simulated electronic prescribing activity for preclerkship medical students.

BACKGROUND: Prescription writing skills are essential for physician practice. This study describes the development and implementation of a curricular intervention focused on improving the knowledge and confidence of preclerkship medical students' prescription writing practices utilizing an interprofessional education model, with a focus on electronic prescribing. METHODS: Medicine and Pharmacy Faculty from a large, urban university collaborated to develop the content of the workshop and a simulation platform was used for the e-prescribing activity. Second-year medical students attended a mandatory in-person workshop facilitated by fourth-year pharmacy students. A pre and post knowledge test and confidence survey were used to assess students' knowledge, confidence, and satisfaction. Outcomes from the knowledge test were evaluated with paired-samples proportions tests, and confidence survey data was evaluated with paired t-tests and Wilcoxon signed-rank tests in a pre-post study design. RESULTS: Students demonstrated a significant increase in prescription writing knowledge and confidence after completing the workshop. On the pre-test, 7% of students (21/284) completed the electronic prescribing assessment correctly and 51% of students (149/295) completed it correctly on the post-test. All items on the confidence survey showed a significant increase in pre- versus post-survey comparisons (p < 0.001). CONCLUSIONS: This interprofessional prescription writing workshop facilitated by pharmacy students shows promise for improving the knowledge and confidence of prescription writing and electronic prescribing practices in preclerkship medical students.

Utility of Kolmogorov complexity measures: Analysis of L2 groups and L1 backgrounds.

The proliferation of automated syntactic complexity tools allowed the analysis of larger amounts of learner writing. However, existing tools tend to be language-specific or depend on segmenting learner production into native-based units of analysis. This study examined the utility of a language-general and unsupervised linguistic complexity metric: Kolmogorov complexity in discriminating between L2 proficiency levels within several languages (Czech, German, Italian, English) and across various L1 backgrounds (N = 10) using two large CEFR-rater learner corpora. Kolmogorov complexity was measured at three levels: syntax, morphology, and overall linguistic complexity. Pairwise comparisons indicated that all Kolmogorov complexity measures discriminated among the proficiency levels within the L2s. L1-based variation in complexity was also observed. Distinct syntactic and morphological complexity patterns were found when L2 English writings were analyzed across versus within L1 backgrounds. These results indicate that Kolmogorov complexity could serve as a valuable metric in L2 writing research due to its cross-linguistic flexibility and holistic nature.

Granting access: Development of a formal course to demystify and promote predoctoral fellowship applications for graduate students.

Strong scientific writing skills are the foundation of a successful research career and require training and practice. Although these skills are critical for completing a PhD, most students receive little formal writing instruction prior to joining a graduate program. In 2015, the University of Iowa Medical Scientist Training Program (MSTP) addressed this issue by developing the scientific writing course Grant Writing Basics (GWB). Here we describe the structure of this course and its effectiveness. GWB is an interactive, workshop-based course that uses a National Institutes of Health (NIH) F30 predoctoral fellowship proposal as a platform for building writing expertise. GWB incorporates established pedagogical principles of adult learning, including flipped classrooms, peer teaching, and reiterative evaluation. Time spent in class centers on active student analysis of previously submitted fellowship applications, discussion of writing resources, active writing, facilitated small group discussion of critiques of student writing samples, revision, and a discussion with a panel of experienced study section members and a student who completed a fellowship submission. Outcomes of GWB include a substantial increase in the number of applications submitted and fellowships awarded. Rigorous evaluation provides evidence that learning objectives were met and that students gained confidence in both their scientific writing skills and their ability to give constructive feedback. Our findings show that investment in formal training in written scientific communication provides a foundation for good writing habits, and the knowledge and skills needed to succeed in this vital aspect of a scientific research career. Furthermore, they highlight that evaluation is valuable in guiding course evolution. Strategies embedded in GWB can be adapted for use in any graduate program to advance scientific writing skills among its trainees.

Good reasons for bad behavior: a randomized controlled experiment on the impact of narrative reading and writing on empathic concern, perspective-taking, and attitude.

Background: Empathic concern and perspective-taking may contribute to avoiding stigmatization of adverse health behavior. Narrative writing has been shown to be effective in promoting perspective-taking and empathy. But since narrative writing is time consuming, we tested in the present study narrative reading as an alternative, more parsimonious approach. Methods: In a randomized controlled experiment, we compared writing a narrative text about a fictitious person who displays disapproved of health behavior to reading such a text and to a control condition in which participants wrote about an unrelated topic. With a sample of n = 194 participants, we investigated the impact of writing and reading a narrative text on promoting empathic concern and perspective-taking as well as on attitude change. Results: We found that both writing and reading a narrative text about the fictitious character increased empathic concern, F(1, 191) = 32.85, p < 0.001, part. η2 = 0.15, and perspective-taking, F(1, 191) = 24.76, p < 0.001, part. η2 = 0.12, more strongly than writing about an unrelated topic. Writing and reading a narrative text also resulted in a more positive attitude toward this person, F(1, 191) = 17.63, p < 0.001, part. η2 = 0.08. Simply reading a narrative text was equally efficient as narrative writing with respect to empathic concern, p = 0.581, perspective-taking, p = 0.629, and attitude, p = 0.197. Conclusion: The finding that narrative reading is as effective as narrative writing suggests that the readers appear to be able to comprehend and engage with the story being told. When narrative reading is as effective as narrative writing, it can succeed with reduced effort in increasing empathic concern, perspective-taking, and attitude. We discuss the benefits of this approach for reducing stigmatization of adverse health behavior.

Association of reviewer experience with discriminating human-written versus ChatGPT-written abstracts.

OBJECTIVE: To determine if reviewer experience impacts the ability to discriminate between human-written and ChatGPT-written abstracts. METHODS: Thirty reviewers (10 seniors, 10 juniors, and 10 residents) were asked to differentiate between 10 ChatGPT-written and 10 human-written (fabricated) abstracts. For the study, 10 gynecologic oncology abstracts were fabricated by the authors. For each human-written abstract we generated a ChatGPT matching abstract by using the same title and the fabricated results of each of the human generated abstracts. A web-based questionnaire was used to gather demographic data and to record the reviewers' evaluation of the 20 abstracts. Comparative statistics and multivariable regression were used to identify factors associated with a higher correct identification rate. RESULTS: The 30 reviewers discriminated 20 abstracts, giving a total of 600 abstract evaluations. The reviewers were able to correctly identify 300/600 (50%) of the abstracts: 139/300 (46.3%) of the ChatGPT-generated abstracts and 161/300 (53.7%) of the human-written abstracts (p=0.07). Human-written abstracts had a higher rate of correct identification (median (IQR) 56.7% (49.2-64.1%) vs 45.0% (43.2-48.3%), p=0.023). Senior reviewers had a higher correct identification rate (60%) than junior reviewers and residents (45% each; p=0.043 and p=0.002, respectively). In a linear regression model including the experience level of the reviewers, familiarity with artificial intelligence (AI) and the country in which the majority of medical training was achieved (English speaking vs non-English speaking), the experience of the reviewer (ß=10.2 (95% CI 1.8 to 18.7)) and familiarity with AI (ß=7.78 (95% CI 0.6 to 15.0)) were independently associated with the correct identification rate (p=0.019 and p=0.035, respectively). In a correlation analysis the number of publications by the reviewer was positively correlated with the correct identification rate (r28)=0.61, p<0.001. CONCLUSION: A total of 46.3% of abstracts written by ChatGPT were detected by reviewers. The correct identification rate increased with reviewer and publication experience.

To Reveal or Not to Reveal, That Is the Wrong Question: Thoughts about Clinical Writing in Psychoanalysis.

This paper plays with the possibilities of writing about psychoanalytic work in different ways with different levels of disclosure about both patient and analyst. Various issues around anonymity, confidentiality, consent and identity are explored, highlighting the many questions that come up. These issues of how to write psychoanalytically are also addressed from the point of view of culture and the sociopolitical gestalt of our time.

What We Do, What We Say, What We Don't Say: Confidentiality In The Publication Of Clinical Writing.

Questions concerning analysts' publication of material from the analyses of their patients have troubled the field of psychoanalysis since its inception. Disguise inevitably distorts the clinical material and is often insufficient to protect the patient from recognition. Asking the patient's consent for publication intrudes upon and alters the analytic process. While analysts have largely reached a consensus about the need for anonymity in published material, there is still considerable debate about the necessity for obtaining patients' consent when using their material for publication. In this paper, I will trace the evolving meanings of disguise, and particularly of consent, in the analytic literature. I will place a particular emphasis upon the differing theoretical belief systems that underlie the analyst's decision to ask consent from her patient or not to do so, and I will argue that, although decisions on asking consent remain a complex matter, such coherent belief systems should play an important part in analysts' decisions regarding consent. I will illustrate my thought processes and some clinical situations with brief examples, and I will conclude with some practical recommendations, with the hope that these will stimulate further discussion in the analytic community.

Scientific writing quality of theses in Medicine and Pharmacy at the University of Bamako (Mali).

AIM: To describe the scientific editorial quality of practice theses defended at the Faculties of Medicine and Pharmacy of Bamako (Mali), during the period 2016-2018. METHODS: This was a cross-sectional study conducted on practice theses in health science, defended between 2016 and 2018, and the electronic version of which was available at the libraries of the Faculties of Medicine and Pharmacy in Bamako. An adjusted STROBE grid was used to measure a scientific editorial conformity score for 18 iso-weighted items of the IMRaD structure (i.e. a total of 18 points). In this way, the scientific editorial quality of practice theses in health sciences was judged to be highly conform (14-18 points), fairly conform (9-13 points), poorly conform (5-8 points), or not conform (<5 points). An ordinal logistic regression model was used to identify determining factors independently associated with the scientific editorial quality of the health sciences practice thesis, with a significance level of 5%. RESULTS: A total of 534 practice theses in health sciences were included in this study, whose scientific editorial quality was judged to be highly or fairly conform (12.2%), poorly conform (43.1%) or not conform (44.8%). These standards scientific editorial conformity, which was higher in a "Medicine" thesis than in a "Pharmacy" thesis (ORa=2.45; IC95% [1.62-3.27], p<=0.05), was attributed to the supervisor's membership of a research structure (ORa=2.88; IC95% [1.6-3.2]; p<0.05). CONCLUSION: The research methodology profile of the practice health sciences thesis supervisor and the thesis section are associated with scientific editorial standard with the IMRaD.